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Program description^ 



SRA Real Math Building Blocks PreK (also referred to as Building 
Blocks for Math) is a supplemental mathematics curriculum 
designed to develop preschool children’s early mathematical 
knowledge through various individual and small- and large- 
group activities. It uses Building Blocks for Math PreK software, 



manipulatives, and print material. Building Blocks for Math 
embeds mathematical learning in children’s daily activities, rang- 
ing from designated math activities to circle and story time, with 
the goal of helping children relate their informal math knowledge 
to more formal mathematical concepts. 



Research Two studies of Building Blocks for Math met the What Works 
Clearinghouse (WWC) evidence standards. ^ They included 
over 250 preschool children from New York State. This report 
focuses on immediate posttest findings to determine the 
effectiveness of the intervention.^ The WWC considers the 



extent of evidence for Building Blocks for Math to be small for 
mathematics achievement. No studies that met WWC evidence 
standards with or without reservations addressed oral language, 
print knowledge, phonological processing, early reading/writing, 
or cognition. 



Effectiveness Building Blocks for Math was found to have positive effects on mathematics achievement. 

Print Phonological Early reading/ 





Oral language 


knowledge processing 


writing 


Cognition 


Math 


Rating of effectiveness 


na 


na 


na 


na 


na 


Positive Effects 


Improvement index'^ 


na 


na 


na 


na 


na 


Average: +36 
percentile points 
Range; +27 to +42 
percentile points 
na = not applicable 



1. The descriptive information for this program was obtained from publicly available sources: the research literature (Clements & Sarama, 2006, June; Cle- 
ments & Sarama, 2007; Clements & Sarama, 2002; Clements & Sarama, n.d.; Sarama & Clements, 2003), and the program description provided by the 
author upon the WWC request. The WWC requests developers to review the program description sections for accuracy from their perspective. Further 
verification of the accuracy of the descriptive information for this program is beyond the scope of this review. 

2. To be eligible for the WWC’s review, the Early Childhood Education (ECE) intervention had to be implemented in English in center-based settings with 
children aged three to five or in preschool. 

3. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

4. These numbers show the average and range of student-level improvement indices for ail findings across the studies. 
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Additional program 
information^ 



WWC Intervention Report 



Developer and contact 

Building Blocks for Math was developed by Drs. Douglas Cle- 
ments and Julie Sarama and is available for preview by contact- 
ing the developers. Address: Graduate School of Education, 
University at Buffalo, The State University of New York, Buffalo, 
NY 14260. Email: clements@buffalo.edu or isarama@buffalo.edu . 
Web: http://www.ase.buffalo.edu/ora/buildinablocks/ . Telephone: 
(716) 645-2455 Ext. 1155. It is distributed by SRA/McGraw-Hill; 
contact Rick Rikhoff, Director of Marketing Services. Email: 
rick rikhoff@mcaraw-hill.com . Telephone: (614) 750-7264. 

Scope of use 

A year after the curriculum was released in 2006, there were 
23,000 children in 51 schools across the nation who purchased 
Building Blocks for Math. In several documented research proj- 
ects, Building Blocks for Math has been implemented by addi- 
tional preschool and childcare programs in New York (more than 
100 classrooms), Massachusetts (more than 60 classrooms), and 
Nashville (more than 50 classrooms). These programs include 
children from low- and mixed-income families. 

Teaching 

The Building Blocks for Math materials integrate three types of 
media: software, manipulatives (and everyday objects), and print 
materials (e.g., books). The materials are designed to be used in a 
variety of environments: home, day care, and classroom. Building 
Blocks for Math integrates computer activities (DLM Express) with 
other activities like teaching games and free-choice learning cen- 
ters. The curriculum is structured around empirically based learn- 
ing trajectories (i.e., what is appropriate for children at that age 



in terms of mathematics). The curriculum includes whole- and 
small-group activities and games, free-choice learning centers, 
ideas for integrating mathematics throughout the school day, 
computer software, and books, game sheets, and manipulatives. 
Activities are designed so they are based on children’s experi- 
ences and interests because the program aims to “mathematize” 
their everyday activities, including building blocks, art, songs, 
stories, and puzzles. The software includes 150 activities (60 are 
intended for preschoolers). The Building Blocks for Math materi- 
als emphasize mathematical actions-on-objects, such as solving 
a sequence of geometric puzzles by filling outlines with pattern 
blocks. 

Teaching guidance is provided for both off- and on-computer 
activities in the teachers’ materials. The software’s management 
system presents tasks, contingent on success along research- 
based trajectories that are sequenced according to children’s 
developmental progression. 

Cost 

Costs for curriculum materials, teacher resources, and software 
are: 

• Real Math Teacher Edition, Pre-K costs $198 

• Real Math Teacher Resource Book, Pre-K costs $129 

• Real Math Assessment Booklet, Pre-K costs $78 

• Real Math Pre-K Big Books (The Shape of Things, I Spy Two 
Eyes, The Right Number of Elephants, and Bat Jamboree) 
each cost $58 

• Real Math Pre-K Manipulative Kit costs $282 

• SRA Math Building Blocks for Math online costs $10 per 
student per year 
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Research Three studies reviewed by the WWC investigated the effects of 
Building Blocks for Math in center-based settings. Two studies 
(Clements & Sarama, 2006; Clements & Sarama, 2007) were 
randomized controlled trials that met WWC evidence standards. 
The remaining study did not meet WWC evidence screens. 

Clements and Sarama (2006) included 28 preschool teachers 
(202 children) from low- to mixed-income families in New York 
State and compared math outcomes for children participating in 
a Building Blocks for Math intervention group to a business-as- 
usual comparison group.® 

Clements and Sarama (2007) included four preschool teachers 
(53 children) from low-income families in New York State and com- 
pared math outcomes for children who participated in a Building 
Blocks for Math group to a business-as-usual comparison group. 



Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or moderate to large (see the What Works Clearinghouse 
Extent of Evidence Categorization Scheme) . The extent of 
evidence takes into account the number of studies and the 
total sample size across the studies that met WWC evidence 
standards with or without reservations.® 

The WWC considers the extent of evidence for Building 
Blocks for Math to be small for mathematics achievement. No 
studies that met WWC evidence standards with or without reser- 
vations addressed oral language, print knowledge, phonological 
processing, early reading/writing, or cognition. 



Effectiveness Findings 

The WWC review of interventions for early childhood education 
addresses children’s outcomes in six domains: oral language, 
print knowledge, phonological processing, early reading/writ- 
ing, cognition, and math. Clements and Sarama (2006, 2007) 
addressed outcomes in the math domain. The findings below 
present the authors’ and the WWC-calculated estimates of the 
size and statistical significance of the effects of Building Blocks 
for Math on children’s performance.^ 

Math. Clements and Sarama (2006) analyzed group differ- 
ences between the Building Blocks for Math intervention group 
and the business-as-usual comparison group on one math out- 



come measure (Early Mathematics Assessment). The difference 
between groups was statistically significant and favored children 
in the Building Blocks for Math group. 

Clements and Sarama (2007) found a statistically significant 
difference favoring children in the Building Blocks for Math 
group on one of the two outcome measures assessed in this 
domain (Building Blocks Assessment of Early Mathematics, 
PreK-K: Geometry section) and this effect was confirmed to be 
statistically significant by the WWC. The authors also reported 
statistically significant differences between the intervention and 
business-as-usual comparison groups on the other math mea- 
sure (Building Blocks Assessment of Early Mathematics, PreK-K: 



5. The study also included a Pre-K Mathematics intervention group, which used DLM Express as an additional component. The study authors labeled the 
Pre-K Mathematics group as the “comparison group” and the Building Blocks for Math group as the “intervention group”; however, the WWC considers 
Pre-K Mathematics as a separate intervention (see the separate WWC Pre-K Mathematics i ntervention reportt . For the rating of effectiveness in this 
WWC intervention report, the WWC includes only the results comparing the Building Blocks for Math group to the business-as-usual comparison group; 
however, results for the comparison between the curricula are included in a separate section of this report and in Appendix A4. 

6. The Extent of Evidence Categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept, external validity, such as the students’ demographics and the types of 
settings in which studies took place, are not taken into account for the categorization. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the Technical Details of WWC-Conducted 
Computations for the formuias the WWC used to calculate the statistical significance. In the case of Building Blocks for Math, corrections for clustering 
and multiple comparisons were needed. 
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Effectiveness (continued) 



Number section): however, the WWC was unable to confirm the 
statistical significance of this effect. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 



effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings,^ the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme) . 



The WWC found Building 
Blocks for Math to have 
positive effects on math 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technical 
Details of WWC-Conducted Computations) . The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition versus 
the percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 
index is based entirely on the size of the effect, regardless of 
the statistical significance of the effect, the study design, or the 
analyses. The improvement index can take on values between 
-50 and +50, with positive numbers denoting results favorable to 
the intervention group. 

The average improvement index for math is +36 percentile 
points across the two studies, with a range of +27 to +42 per- 
centile points across findings. 

Findings for comparisons between Building Blocks for Math 
and Pre-K Mathematics 

The analysis for the comparison described below was included 
in the Clements and Sarama (2006) study, but the findings do not 
contribute to the overall rating of effectiveness because the WWC 
used the comparison of Building Blocks for Math to the business- 
as-usual comparison group from the same study in the rating, 



which provides a clearer sense of Building Blocks for Math’s 
effects. However, the WWC believes that the findings from this 
comparison provide useful information to practitioners who may 
be interested in comparing the effects of different curricula. The 
WWC reports the findings for comparisons of Building Blocks for 
Math and Pre-K Mathematics here and in Appendix A4. 

Math. Clements and Sarama (2006) analyzed group differ- 
ences between the Building Blocks for Math group and the 
Pre-K Mathematics combined with DLM Express group for 
one math outcome measure (Early Mathematics Assessment). 
The difference between groups was statistically significant and 
favored children in the Building Blocks for Math group. The 
improvement index for math is +19 percentile points [Building 
Blocks for Math is the intervention group and Pre-K Mathematics 
is the comparison group) for the single finding in the study. 

Summary 

The WWC reviewed three studies on Building Blocks for Math. 
Two of these studies met WWC evidence standards; the remain- 
ing study did not meet WWC evidence screens. Based on these 
two studies, the WWC found positive effects for math. Additional 
findings that were not considered for the rating of effectiveness 
indicated that Building Blocks for Math may have a larger impact 
on children’s math outcomes than another skills-focused pre- 
school mathematics intervention. The evidence presented in this 
report may change as new research emerges. 
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Appendix 



Appendix A1.1 


Study characteristics: Ciements & Sarama, 2006 (randomized controiied triai) 


Characteristic 


Description 


Study citation 


Clements, D. H., & Sarama, J. (2006, June). Scaling up the implementation of a pre-Kindergarten mathematics curriculum: The Building Blocks curriculum. Paper presented at 
the Institute of Education Sciences Research Conference, Washington, D.C. 


Participants 


Teachers were randomly assigned to conditions in two separate steps. Twenty-four teachers from preschool programs serving low-income children were randomly assigned to 
two intervention groups (Building Blocks for Math or Pre-K MathematicsY or a business-as-usual comparison group. Twelve teachers from preschool programs serving mixed- 
income children were randomly assigned to the Building Blocks for Math group or the business-as-usual comparison group. Consequently there were a total of 14 teachers in 
the Building Blocks for Math group and 14 teachers in the business-as-usual comparison group. Eight preschool-age children were randomly selected from each classroom 
for assessment (N = 224).^ After attrition, the final sample included 28 teachers and 202 children (14 teachers and 101 children in the Building Blocks for Math Qmup\ 14 
teachers and 101 children in the business-as-usual comparison group). 


Setting 


The study was conducted in Head Start and state-funded preschool programs in New York State. 


Intervention 


Children in the Building Blocks for Math Menenfm condition used the Building Blocks for Math mnculum in 10- to 15-minute small-group (4-6 children) math activities 
weekly. These children also participated in 5- to 15-minute whole-group math activities four times a week and 5- to 10-minute computer activities (DLM Express) W\ce a 
week. Related family activities were sent home weekly. The intervention lasted for 26 weeks, and intervention teachers maintained their daily activities and schedule while 
inserting mathematics activities at appropriate times during the day.^ 


Comparison 


Children in the business-as-usual comparison group participated in their regular daily activities and schedule, with emphasis on small groups and computer activities. These 
included city-wide math activities. Creative Curriculum, Montessori math activities, or “home-grown” math materials based on state standards. 


Primary outcomes 
and measurement 


The primary outcome domain assessed was math and it was measured with the Early Mathematics Assessment (see Appendix A2 for a more detailed description of the 
outcome measure). The study authors also assessed implementation fidelity with the Eidelity of Implementation measure and the quality of the mathematics environment using 
the Classroom Observation of Early Mathematics Environment and Teaching. This WWC review does not include the results from these observations in this WWC review.'* 


Teacher training 


Professional development activities for teachers in the Building Blocks for Math group consisted of four days of training, a monthly two-hour class, and monthly in-class coach- 
ing by project staff. Teacher training covered a number of topics such as supporting mathematical development in the classroom, recognizing and supporting mathematics 
throughout the day, setting up mathematics learning centers, teaching with computers, small-group activities, and supporting mathematical development in the home. Learn- 
ing trajectories were emphasized in the Building Blocks for /Waft training via a web-based application called Building Blocks Learning Trajectories. 



1 . The study also included a Pre-K Mathematics intervention group, which used DLM Express as an additional component. The study authors labeled the Pre-K Mathematics group as the 
“comparison group” and the Buiiding Biocks for Math group as the “intervention group”; however, the WWC considers Pre-K Mathematics as a separate intervention (see the separate WW C 
Pre-K Mathematics intervention report). For the rating of effectiveness in this WWC intervention report, the WWC includes only the results comparing the Buiiding Biocks for Math group to the 
business-as-usual comparison group; however, results for the comparison between the curricula are included in Appendix A4. 

2. The remaining eight teachers were assigned to the Pre-K Mathematics group, which is not the main focus of this WWC intervention report. 

3. Children in the Pre-K Mathematics intervention group participated in 15- to 20-minute small-group activities at least twice a week and 10- to 15-minute whole-class math activities once a week. 
In addition, these classrooms used DLM Express software for 5 to 10 minutes twice a week. Weekly letters were sent to parents that included math activities similar to those children were learn- 
ing at school. The intervention lasted for 26 weeks and the teachers maintained their daily activities and schedule while inserting mathematics activities at appropriate times during the day. 

4. For further details about the outcomes included in the Early Childhood Education topic review, please see the Early Childhood Education Protocol . 
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Appendix A1.2 Study characteristics: Ciements & Sarama, 2007 (randomized controiied triai)^ 



Characteristic 


Description 


Study citation 


Clements, D. H., & Sarama, J. (2007). Effects of a preschool mathematics curriculum: Summative research on the Building Blocks project. Journal for Hesearch in Mathemat- 
ics Education, 38(2), 136-163. 

Additional sources: 

Clements, D. H., & Sarama, J. (2002). Effects of a preschool mathematics curriculum: Research on the NSE-funded Building Blocks Project University at Buffalo, State 
University of New York. 

Clements, D. H., & Sarama, J. (n.d.). Effects of a preschool mathematics curriculum: Summary research on the NSE-funded Building Blocks project University at Buffalo, 
State University of New York. Retrieved from http://www.gse.buffalo.edu/org/buildingblocks/writings/Building%20Blocks%20Research%201 .pdf 


Participants 


Two teachers from a Head Start center and two teachers from a state-funded preschool program were randomly assigned to the Building Blocks for Math group or the 
business-as-usual comparison group. There were 77 children in the four classrooms; however, nine children left the classrooms during the course of the study. Final analysis 
samples ranged from 53 to 61 children.^ The majority of the children were from low-income families. The mean age of the children was 49.9 months, and 49% of the children 
were female. 


Setting 


The study took place in one Head Start and one state-funded preschool program in New York State. 


Intervention 


The intervention group used Building Blocks for Math tor 25 weeks. Building Blocks for Math includes whole- and small-group activities and games, free-choice learning 
centers, ideas for integrating mathematics throughout the school day, computer software (DLM Express), and books, game sheets, and manipulatives. Teachers in the 
state-funded preschool program implemented the curriculum with a high degree of fidelity, whereas teachers in the Head Start center implemented it with a moderate — but 
adequate — degree of fidelity. 


Comparison 


Children in the business-as-usual comparison group used their typical preschool math curricula and activities over the same 25 weeks. The typical preschool math curriculum 
used in the Head Start comparison classroom was Creative Cr/mcu/um and math activities developed by the program. The state-funded preschool program comparison 
classroom continued implementing the school’s regular math curriculum. 


Primary outcomes 
and measurement 


The primary outcome domain assessed was math, and it was measured with the Number and Geometry sections of the Building Blocks Assessment of Early Mathematics, 
PreK-K (see Appendix A2 for more detailed descriptions of outcome measures). 


Teacher training^ 


Intervention teachers participated with researchers in a half-day review of curriculum materials. Project staff observed intervention teachers implementing the curriculum and 
were available to answer questions and discuss implementation issues with teachers as needed. 



1 . According to data provided by the study authors upon the WWC request, there was within-ciuster attrition of 21.6% between the intervention (10% of children) and business-as-usuai compari- 
son (31.6% of chiidren) groups. However, the WWC did not downgrade the study because there was pretest equivaience between groups for the non-attritors. 

2. The study authors had worked previousiy with the teachers at the stated-funded preschooi program, the intervention teacher at the Head Start was inexperienced, and the comparison teacher 
at Head Start had taught Head Start for eight years. 
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Appendix A2 Outcome measures in the math domain^ 



Outcome measure 


Description 


Early Mathematics 
Assessment 


A researcher-developed measure that uses two individual child interviews to assess young children’s mathematical knowledge and skills in the areas of number, geometry, 
measurement, and patterning (as cited in Clements & Sarama, 2006). 


Building Blocks Assessment 
of Early Mathematics, 
PreK-K; Number section 


A researcher-developed measure that uses individual child interviews to assess young children’s mathematical knowledge. The number section measures nine topics, including 
verbal counting, object counting, number recognition, number comparison, number sequencing, numerals, number composition, adding and subtracting, and place value (as 
cited in Clements & Sarama, 2007). 


Building Blocks Assessment 
of Early Mathematics, 
PreK-K; Geometry section 


A researcher-developed measure that uses individual child interviews to assess young children’s mathematical knowledge. The geometry section measures seven topics, 
including shape identification, shape composition, congruence, construction of shape, turns, measurement, and patterning (as cited in Clements & Sarama, 2007). 



1. The Early Mathematics Assessment is an earlier version of the Building Blocks Assessment of Early Mathematics. The name change was at the publisher’s request. 
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Appendix A3 Summary of study findings inciuded in the rating for the math domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
children)^ 


Building Blocks 

for Math Comparison 

group'* group'* 


Mean difference^ 
{Building Blocks 
for Math - 
comparison) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index® 



Early Mathematics 
Assessment 


Preschool children 


Clements & Sarama, 2006 (randomized controlled trial)® 

28/202 61.69 53.22 8.47 

(7.46) (8.38) 


1.06 


Statistically 

significant 


-f36 


Average*® for math (Clements & Sarama, 2006) 










1.06 


Statistically 

significant 


-f36 






Clements & Sarama, 2007 (randomized controlled trial)** 








BBA: Number section 


4 year olds 


4/61 


28.23 


17.93 


10.30 


0.75 


ns 


-f27 








(14.47) 


(12.48) 










BBA: Geometry section 


4 year olds 


4/53 


15.75 


10.64 


5.11 


1.40 


Statistically 


-f42 








(3.81) 


(3.35) 






significant 




Average*® for math (Clements & Sarama, 2007) 










1.08 


Statistically 


-f36 
















significant 




Domain average*® for math across all studies 










1.07 


na 


-f36 



BBA = Building Blocks Assessment of Early Mathematics, PreK-K 

ns = not statistically significant 

na = not applicable 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices. Findings from fhe head-to-head comparison oi Building Blocks for Math to Pre-K Mathematicsfmm Clements and Sarama 
(2006) are not included in these ratings, but are reported in Appendix A4. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The standard 
deviations for Clements and Sarama (2006) were provided by the study authors upon WWC request. 

3. The child-level sample sizes were provided by the study authors upon WWC request. 

4. For Clements and Sarama (2006), the intervention group mean (61 .69) equals the comparison group mean (53.22) plus the program coefficient from the author-conducted HLM analysis (8.47). For Clements and Sarama (2007), the 
intervention group mean equals the comparison group mean plus the mean difference. 

5. Positive differences and effecf sizes favor fhe intervenfion group; negafive differences and effecf sizes favor fhe comparison group. For Clemenfs and Sarama (2006), fhe mean difference was the program coefficient from fhe aufhor- 
conducfed HLM analysis. For Clements and Sarama (2007), the mean differences were compufed by fhe WWC and took into account pretest difference befween fhe sfudy groups. The resulfing effecf sizes may overesfimafe fhe 
infervenfion's effecfs when the intervention group had lower pretest scores than the comparison group and underestimate the intervention’s effects when the intervention group had higher pretest scores than the comparison group. 

6. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition versus the percentile rank of the average student in the comparison condition. The improvement index 

can take on values between -50 and -t-50, with positive numbers denoting results favorable to the intervention group. (continued) 
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Appendix A3 Summary of study findings inciuded in the rating for the math domain^ (continued) 



9. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Clements and Sarama (2006), no cor- 
rections for clustering or multiple comparisons were needed. 

10. The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. 

1 1 . In the case of Clements & Sarama (2007), corrections for clustering and multiple comparisons were needed, so the significance levels differ from those reported in the original study. 
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Appendix A4 Summary of findings for comparisons between Building Blocks for Math and Pre-K Mathematics for the math domain^ 









Authors’ findings from the study 














Mean outcome 
(standard deviation^) 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(teachers/ 
children)^ 


Building Blocks 

for Math Pre-K Math 

group'* group'* 


Mean difference^ 
{Building Blocks 
for Math - 
Pre-K Math) 


Statistical 
significance^ 
Effect size® (at a = 0.05) 


Improvement 

index® 





Clements & Sarama, 2006 (randomized controlled trial)® 








Early Mathematics Assessment Preschool children 


21/152 57.32 53.77 3.55 

(7.46) (6.53) 


0.49 


Statistically 

significant 


-f19 


Domain average*® for math 




0.49 


Statistically 

significant 


-f19 



BBA = Building Blocks Assessment of Early Mathematics, PreK-K 

1. This appendix presents findings for the head-to-head comparison of Building Blocks for Math an6 Pre-K Mathematics for a measure fhaf falls in fhe mafh domain. Pre-K Mathematicswas implemenfed in conjuncfion wifh an addifional 
componenf (DLM Express soffware). Comparisons of Building Blocks for Math and fhe business-as-usual comparison group were used for rafing purposes and are presenfed in Appendix A3. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. The standard 
deviations were provided by the study authors upon WWC request. 

3. The child-level sample size was provided by the study authors upon WWC request. 

4. The Building Blocks for Math group mean (57.32) equals the Pre-K Mathematics group mean (53.77) plus the program coefficient from the author-conducted HLM analysis (3.55). 

5. Positive differences and effect sizes favor the Building Blocks for Math group; negative differences and effect sizes favor the Pre-K Mathematics group. The mean difference was the program coefficient from the author-conducted HLM 
analysis. 

6. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations . 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

8. The improvement index represents the difference between the percentile rank of the average student in the Building Blocks for Math condition versus the percentile rank of the average student in the Pre-K Mathematics condition. The 
improvement index can take on values between -50 and -t-50, with positive numbers denoting results favorable to the Building Blocks for Math group. 

9. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Clements and Sarama (2006), no cor- 
rections for clustering or multiple comparisons were needed. 

1 0. This row provides the study average, which in this instance, is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated 
from the average effect size. 



(continued) 
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Appendix A5 Building Blocks for Math rating for the math domain 



The WWC rates an intervention’s effects in a given outcome domain as positive, potentiaily positive, mixed, no discernibie effects, potentialiy negative, or negative.'' 

For the outcome domain of math, the WWC rated Building Blocks for Math as having positive effects. The remaining ratings (potentiaiiy positive effects, mixed 
effects, no discernibie effects, potentialiy negative effects, and negative effects) were not considered because Building Blocks for Math was assigned the highest 
appiicabie rating. 



Rating received 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statisticaiiy significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Met. Two studies of Building Blocks for Math showed statisticaiiy significant positive effects and both studies met WWC evidence standards for a 
strong design. 

AND 

• Criterion 2: No studies showing statisticaiiy significant or substantively important negative effects. 

Met. No studies showed statisticaiiy significant or substantively important negative effects. 

1 . For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. 
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Appendix A6 Extent of evidence by domain 



Outcome domain 


Number of studies 


Schoois 


Sample size 

Classrooms/children 


Extent of evidence^ 


Oral language 


0 


0 




0 


na 


Print knowledge 


0 


0 




0 


na 


Phonological processing 


0 


0 




0 


na 


Early reading/writing 


0 


0 




0 


na 


Cognition 


0 


0 




0 


na 


Math 


2 


4+ 




32/255 


Small 



na = not applicable/not studied 

1 . A rating of “moderate to large” requires at least two studies and two schools across studies in one domain, and a total sample size across studies of at least 350 students or 14 classrooms. 
Otherwise, the rating is “small.” 
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